Digital non-linear editing (DNLE) is a process by which digital media may be edited. DNLE, as the name implies, is performed on digital media stored as data in digital media files on a digital random access medium. DNLE may be conducted in a non-linear fashion because the digital media files in which the digital media is stored can be randomly accessed. Thus an editor may access a piece of the digital media without having to proceed sequentially through other pieces of the digital media stored in the same or other digital media files. More than one editor also may be able to access different pieces of the same digital media contemporaneously. The digital media may be a digitized version of a film or videotape or digital media produced through live capture onto a disk of a graphics or animation software application. Example commercial DNLE systems include the Media Composer(copyright) or the Avid(copyright) Symphony(trademark) video production system or, NewsCutter(copyright) news editing system available from Avid Technology, Inc. For a more detailed description of DNLE, see Digital Nonlinear Editing, New Approaches to Editing Film and Video, 1993, by Thomas Ohanian.
Color modification is a class of operations that may be performed on media data. Color modification is used both to correct color errors due to process errors and to adjust the colors used in the video for artistic expression. Such color modifications may include enhancing contrasts or color in an image to give a program an overall xe2x80x9clook,xe2x80x9d or applying special effects to selected segments. Other color modifications may be made by an editor to correct problems with color or lighting resulting from the source of the media. Such corrections may include color balancing for camera and lighting differences, correcting for film processing differences, matching colors and tones from shot to shot, or adjusting video levels for differences in source tapes, source decks, etc.
Digital images are comprised of an array of picture elements called pixels. For a given image, color modifications may be applied to all pixels in the image or pixels comprising a portion of the image. In digital video signal processing, a variety of data formats can be used to represent the color of pixels within a digital image. Formats may be classified into two major categories: composite signals and component signals. Component formats represent a color as multiple components, each component defining a value along a dimension of the color space in which the color being represented is defined. A composite video is an analog signal that uses a high frequency subcarrier to encode color information. The subcarrier is a sinewave of which the amplitude is modulated by the saturation of the color represented by the signal, and the hue of the color is encoded as a phase difference from a color burst. Analog composite signals are generally used to broadcast television video signals.
There are a variety of component formats used to represent color. RGB (Red, Green, Blue) format represents a color with a red component, a green component and a blue component. CMY (Cyan, Magenta, Yellow, Black) format represents a color with a cyan component, a magenta component, and a yellow component. CMY is a format commonly used by printers. The CMY components are color opposites of RGB components. In a three-dimensional coordinate system, each component of either the RGB or the CMY format represents a value along an axis, the combination of the values defining a cubic color space.
The data formats HSL (Hue, Saturation, Lightness or Luminance) and HSV (Hue, Saturation, Value) represent a color with a hue component, a saturation component, and a luma component. In a three-dimensional coordinate system, the luma component represents a value along a luma axis, the hue component represents the angle of a chroma vector with respect to the luma axis and the saturation component represents the magnitude of the chroma vector. The combination of the values defines a hexagonal cone-shaped color space.
YCrCb, YUV, and YIQ are three formats that represent a color with a luma component Y, and two chroma components, Cr and Cb, U and V, or I and Q, respectively, that define a chroma vector. In a three-dimensional coordinate system, each component of either the YCrCb, YUV, and YIQ format represents a value along an axis, the combination of the values defining a cylindrical color space around the luma axis. The chroma components define the chroma vector. In data formats with a luma component, the luma component can be used independently to represent a pixel in a black and white image to be displayed, for example, with a black and white monitor.
A typical color modification in HSL color space may include increasing a color component or a combination of color components for all pixels in each digital image of a section of digital media. Typically, an editor accesses a segment of a composition that represents the section of media through an editor interface and inputs desired color modifications through the editor interface. Some systems permit an editor to apply color modifications to only portions of a digital image. Portions of a digital image also may be described as one or more pixels. For example, an editor may select with a mouse, keyboard, or some other editor input device a portion of the image and define color modifications for the selected portion. A suitable commercial system for color modification is Avid Media Illusion(trademark) available from Avid Technology, Inc. The Avid Media Illusion Reference Guide, available from Avid Technology, Inc. is herein incorporated by reference. Other commercial software applications may be used.
Some systems permit an editor to define a color modification as a function of the luma of a pixel. Some systems also allow a user to define functions of luma that allow an editor to define the effect of a color modification over a range of possible luma values of a pixel. For example, an editor may define a highlight function that primarily affects high luma values, a midtone function that primarily affects mid-range luma values, and a shadow function that primarily affects low luma values. The editor may then associate a color modification with each function. If more than one function is defined over a range of luma values, each function, and thus a color modification associated with the function, may have a weighted effect for a given luma value. This weighted effect is defined by the value of the function for the given luma value normalized with respect to the values of functions for the given luma value. Although luma may be used to determine the weighting of the color modification, the color modification itself is not limited to adjusting the luma. Typical color modifications include adjustments to hue, saturation, luminance, brightness, contrast, gamma, tinting, etc.
For example, a color modification A is specified for a luma function defined by X=2Lxe2x88x922, and a color modification B is specified for a luma function defined by Y=0.5L+2. For pixel P with a luma value of 16, X=30 and Y=10. Normalizing X and Y, the weighted value of X is 0.75 and the weighted value of Y is 0.25. Thus, the color modification applied to pixel P is Z=0.75A+0.25B. The function used in this example is linear to simplify the explanation, but for decreasing and increasing the effect of a color modification for different values of luma, the function is usually non-linear.
The range of possible luma values may have ranges in which one of the functions has a predominant effect. For the highlight, midtone, and shadow functions described above, a highlight range is the range of luma values for which the highlight function has the strongest effect or the greatest weighted value, a midtone range is the range of luma values for which the midtone function has the strongest effect or the greatest weighted value, and the shadow range is the range of luma values for which the shadow function has the strongest effect or the greatest weighted value.
Some systems allow an editor to view a visual representation of the effect of more than one luma function on a digital image, where a selected luma function is emphasized. This representation generally is a substantially continuous gray scale image. In the gray scale image, pixels represented as white are pixels having a luma value for which the selected function has the strongest contribution. Pixels represented as black are pixels having a luma value for which the selected luma function has no contribution. All other pixels are represented as levels of gray between white and black, where the gray level representing the pixel is inversely proportional to the weighted value of the luma function for the luma value of the pixel. A suitable commercial product for defining luma functions, defining color modifications for each function, and viewing a representation of an image emphasizing a selected luma function is Avid Media Illusion (trademark) available from Avid Technology, Inc.
A problem with systems allowing an editor to define luma functions and color modifications for each luma function is that an editor cannot always determine from a visual inspection of a digital image which pixels or portions of the image fall within a luma range. Thus the editor may not know which pixels will be most affected by a color modification defined for a function. Even if an editor may view the effects of more than one function on a digital image in gray scale, with a selected function emphasized, this gray scale image does not illustrate the boundaries of the luma ranges, and thus does not give the editor a clear illustration of which function has the strongest effect for a range of luma values. The gray scale image also emphasizes only one luma function at a time. For a selected function, the gray scale image also may be confusing and not intuitive because the gray scale image may not represent the pixels in proportion to their luma. For example, if a midtone function is selected, pixels having a value for which the midtone function has the highest weighted value are represented as white, which has high luma, and pixels affected more by a highlight function, usually pixels having higher luma values, are represented as darker grays which have lower luma. For an editor, especially a novice to color modification, this confusion may result in a lack of efficiency.
Providing a multi-tone representation of a digital image, where each tone represents a luma range, permits an editor to visually identify pixels of a digital image that fall within predefined luma ranges. This multi-tone representation allows an editor to control the effect of luma-based color modifications to pixels in a more efficient manner.
Multi-tone representation of a digital image provides a method of producing a representation of a digital image by applying a plurality of functions of luma to the digital image. The luma functions may be selected and redefined by a user. The functions of luma comprise at least a first function and a second function, where each function has a value for each luma value in a range of luma values. Each function also has an associated display value. The digital image includes a plurality of pixels, where for each pixel, it is determined which of the first and second functions has a maximum value for a luma value of the pixel. Each pixel is represented using the display value associated with the determined function. The representation of the digital image may be displayed on a display device. Each pixel of the representation may be displayed as the level of gray represented by the display value determined for each pixel.
In one embodiment, each display value associated with one of the functions represents a level of gray. In an alternative embodiment, each display value associated with one of the functions represents a color.